{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "1D convnet_JeenaClimate.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "0rj4xoEI8PGs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        },
        "outputId": "b9a5558a-e629-49ed-97d1-e05535eaeb91"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "62Drd5JI8138",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 345
        },
        "outputId": "177bc724-1d60-433d-aaab-89df59102765"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "df = pd.read_csv(\"/content/drive/My Drive/ML and AI/Datasets/jena_climate/jena_climate_2009_2016.csv\")\n",
        "print(\"Length of dataset :\",len(df))\n",
        "print()\n",
        "display(df.head())\n",
        "values = df.iloc[:,1:] # excluded date\n",
        "float_data = np.asarray(values) # converting in numpy array"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Length of dataset : 420551\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Date Time</th>\n",
              "      <th>p (mbar)</th>\n",
              "      <th>T (degC)</th>\n",
              "      <th>Tpot (K)</th>\n",
              "      <th>Tdew (degC)</th>\n",
              "      <th>rh (%)</th>\n",
              "      <th>VPmax (mbar)</th>\n",
              "      <th>VPact (mbar)</th>\n",
              "      <th>VPdef (mbar)</th>\n",
              "      <th>sh (g/kg)</th>\n",
              "      <th>H2OC (mmol/mol)</th>\n",
              "      <th>rho (g/m**3)</th>\n",
              "      <th>wv (m/s)</th>\n",
              "      <th>max. wv (m/s)</th>\n",
              "      <th>wd (deg)</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>01.01.2009 00:10:00</td>\n",
              "      <td>996.52</td>\n",
              "      <td>-8.02</td>\n",
              "      <td>265.40</td>\n",
              "      <td>-8.90</td>\n",
              "      <td>93.3</td>\n",
              "      <td>3.33</td>\n",
              "      <td>3.11</td>\n",
              "      <td>0.22</td>\n",
              "      <td>1.94</td>\n",
              "      <td>3.12</td>\n",
              "      <td>1307.75</td>\n",
              "      <td>1.03</td>\n",
              "      <td>1.75</td>\n",
              "      <td>152.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>01.01.2009 00:20:00</td>\n",
              "      <td>996.57</td>\n",
              "      <td>-8.41</td>\n",
              "      <td>265.01</td>\n",
              "      <td>-9.28</td>\n",
              "      <td>93.4</td>\n",
              "      <td>3.23</td>\n",
              "      <td>3.02</td>\n",
              "      <td>0.21</td>\n",
              "      <td>1.89</td>\n",
              "      <td>3.03</td>\n",
              "      <td>1309.80</td>\n",
              "      <td>0.72</td>\n",
              "      <td>1.50</td>\n",
              "      <td>136.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>01.01.2009 00:30:00</td>\n",
              "      <td>996.53</td>\n",
              "      <td>-8.51</td>\n",
              "      <td>264.91</td>\n",
              "      <td>-9.31</td>\n",
              "      <td>93.9</td>\n",
              "      <td>3.21</td>\n",
              "      <td>3.01</td>\n",
              "      <td>0.20</td>\n",
              "      <td>1.88</td>\n",
              "      <td>3.02</td>\n",
              "      <td>1310.24</td>\n",
              "      <td>0.19</td>\n",
              "      <td>0.63</td>\n",
              "      <td>171.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>01.01.2009 00:40:00</td>\n",
              "      <td>996.51</td>\n",
              "      <td>-8.31</td>\n",
              "      <td>265.12</td>\n",
              "      <td>-9.07</td>\n",
              "      <td>94.2</td>\n",
              "      <td>3.26</td>\n",
              "      <td>3.07</td>\n",
              "      <td>0.19</td>\n",
              "      <td>1.92</td>\n",
              "      <td>3.08</td>\n",
              "      <td>1309.19</td>\n",
              "      <td>0.34</td>\n",
              "      <td>0.50</td>\n",
              "      <td>198.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>01.01.2009 00:50:00</td>\n",
              "      <td>996.51</td>\n",
              "      <td>-8.27</td>\n",
              "      <td>265.15</td>\n",
              "      <td>-9.04</td>\n",
              "      <td>94.1</td>\n",
              "      <td>3.27</td>\n",
              "      <td>3.08</td>\n",
              "      <td>0.19</td>\n",
              "      <td>1.92</td>\n",
              "      <td>3.09</td>\n",
              "      <td>1309.00</td>\n",
              "      <td>0.32</td>\n",
              "      <td>0.63</td>\n",
              "      <td>214.3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             Date Time  p (mbar)  T (degC)  ...  wv (m/s)  max. wv (m/s)  wd (deg)\n",
              "0  01.01.2009 00:10:00    996.52     -8.02  ...      1.03           1.75     152.3\n",
              "1  01.01.2009 00:20:00    996.57     -8.41  ...      0.72           1.50     136.1\n",
              "2  01.01.2009 00:30:00    996.53     -8.51  ...      0.19           0.63     171.6\n",
              "3  01.01.2009 00:40:00    996.51     -8.31  ...      0.34           0.50     198.0\n",
              "4  01.01.2009 00:50:00    996.51     -8.27  ...      0.32           0.63     214.3\n",
              "\n",
              "[5 rows x 15 columns]"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SnlcU5D09SPk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Normalizing data\n",
        "\n",
        "mean = float_data[:200000].mean(axis=0)\n",
        "float_data -= mean\n",
        "std = float_data[:200000].std(axis=0)\n",
        "float_data /= std"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OkRGGpsY9a6Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Making genrator\n",
        "\n",
        "def generator(data, lookback, delay, min_index, max_index,shuffle=False, batch_size=128, step=6):\n",
        "    if max_index is None: #If max index is not specified, we will set it to very end of the data\n",
        "        max_index = len(data) - delay - 1 # 1 step peche kharehoke agay ka data pridict kara sakte hai\n",
        "    i = min_index + lookback # 0 se start nahe kersakte lookback kerna hai to utna agay se stat kerna parega\n",
        "    while 1:\n",
        "        if shuffle: \n",
        "            #Generate random rows\n",
        "            rows = np.random.randint(min_index + lookback, max_index, size=batch_size)\n",
        "        else:\n",
        "            if i + batch_size >= max_index: \n",
        "                i = min_index + lookback    # ager data khatam ho raha ho to yeah set kerdo\n",
        "            rows = np.arange(i, min(i + batch_size, max_index))\n",
        "            i += len(rows) # \n",
        "        samples = np.zeros((len(rows),lookback // step,data.shape[-1])) # past data\n",
        "        targets = np.zeros((len(rows),))\n",
        "        for j, row in enumerate(rows):\n",
        "            indices = range(rows[j] - lookback, rows[j], step)\n",
        "            samples[j] = data[indices]\n",
        "            targets[j] = data[rows[j] + delay][1]\n",
        "        yield samples, targets # sample pe lookback aur target mai delay ka bht sara data hai"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o4ZQ0VMe9iKU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lookback = 1440   # 1440/6 = 240(samples) kitna past data dekhna hai\n",
        "step = 6          # 10 min bt ka data hai, 6 step matlab 60 min(1 hour) ke bt ka data lo\n",
        "delay = 144\n",
        "batch_size = 128  # kitne line uthane hai\n",
        "\n",
        "train_gen = generator(float_data,\n",
        "                        lookback=lookback,\n",
        "                        delay=delay,\n",
        "                        min_index=0,\n",
        "                        max_index=200000,\n",
        "                        shuffle=True,\n",
        "                        step=step,\n",
        "                        batch_size=batch_size)\n",
        "\n",
        "val_gen = generator(float_data,\n",
        "                    lookback=lookback,\n",
        "                    delay=delay,\n",
        "                    min_index=200001,\n",
        "                    max_index=300000,\n",
        "                    step=step,\n",
        "                    batch_size=batch_size)\n",
        "\n",
        "test_gen = generator(float_data,\n",
        "                    lookback=lookback,\n",
        "                    delay=delay,\n",
        "                    min_index=300001,\n",
        "                    max_index=None,\n",
        "                    step=step,\n",
        "                    batch_size=batch_size)\n",
        "\n",
        "val_steps = (300000 - 200001 - lookback) // batch_size # in book code batch size was not divided\n",
        "# How many steps to draw from val_gen in order to see the entire validation set\n",
        "\n",
        "test_steps = (len(float_data) - 300001 - lookback) // batch_size # in book code batch size was not divided\n",
        "# How many steps to draw from test_gen in order to see the entire test set"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7dYuACU9wB0",
        "colab_type": "text"
      },
      "source": [
        "# Training and evaluating a simple 1D convnet on the Jena data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Oo1qvTsM9sH0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 399
        },
        "outputId": "14366803-15a6-410f-e1c4-cb6608565579"
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.optimizers import RMSprop\n",
        "\n",
        "model = Sequential()\n",
        "model.add(layers.Conv1D(32, 5, activation='relu',input_shape=(None, float_data.shape[-1])))\n",
        "model.add(layers.MaxPooling1D(3))\n",
        "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
        "model.add(layers.MaxPooling1D(3))\n",
        "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
        "model.add(layers.GlobalMaxPooling1D())\n",
        "model.add(layers.Dense(1))\n",
        "\n",
        "model.compile(optimizer=RMSprop(), loss='mae')\n",
        "model.summary()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv1d_3 (Conv1D)            (None, None, 32)          2272      \n",
            "_________________________________________________________________\n",
            "max_pooling1d_2 (MaxPooling1 (None, None, 32)          0         \n",
            "_________________________________________________________________\n",
            "conv1d_4 (Conv1D)            (None, None, 32)          5152      \n",
            "_________________________________________________________________\n",
            "max_pooling1d_3 (MaxPooling1 (None, None, 32)          0         \n",
            "_________________________________________________________________\n",
            "conv1d_5 (Conv1D)            (None, None, 32)          5152      \n",
            "_________________________________________________________________\n",
            "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 1)                 33        \n",
            "=================================================================\n",
            "Total params: 12,609\n",
            "Trainable params: 12,609\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3ngYN1a_98oE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 784
        },
        "outputId": "b7bbc63a-4b90-480c-c62b-b91f59801e31"
      },
      "source": [
        "history = model.fit_generator(train_gen,\n",
        "                              steps_per_epoch=500,\n",
        "                              epochs=20,\n",
        "                              validation_data=val_gen,\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From <ipython-input-8-4f77649be1a6>:5: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use Model.fit, which supports generators.\n",
            "Epoch 1/20\n",
            "500/500 [==============================] - 12s 25ms/step - loss: 0.4140 - val_loss: 0.4744\n",
            "Epoch 2/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.3571 - val_loss: 0.4660\n",
            "Epoch 3/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.3351 - val_loss: 0.4500\n",
            "Epoch 4/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.3174 - val_loss: 0.4720\n",
            "Epoch 5/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.3035 - val_loss: 0.4681\n",
            "Epoch 6/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2942 - val_loss: 0.4740\n",
            "Epoch 7/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2855 - val_loss: 0.4581\n",
            "Epoch 8/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2792 - val_loss: 0.4931\n",
            "Epoch 9/20\n",
            "500/500 [==============================] - 12s 23ms/step - loss: 0.2736 - val_loss: 0.4714\n",
            "Epoch 10/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2677 - val_loss: 0.4644\n",
            "Epoch 11/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2642 - val_loss: 0.4697\n",
            "Epoch 12/20\n",
            "500/500 [==============================] - 12s 25ms/step - loss: 0.2617 - val_loss: 0.4651\n",
            "Epoch 13/20\n",
            "500/500 [==============================] - 12s 23ms/step - loss: 0.2565 - val_loss: 0.4704\n",
            "Epoch 14/20\n",
            "500/500 [==============================] - 12s 23ms/step - loss: 0.2525 - val_loss: 0.4887\n",
            "Epoch 15/20\n",
            "500/500 [==============================] - 12s 23ms/step - loss: 0.2509 - val_loss: 0.4577\n",
            "Epoch 16/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2482 - val_loss: 0.4692\n",
            "Epoch 17/20\n",
            "500/500 [==============================] - 13s 25ms/step - loss: 0.2474 - val_loss: 0.4661\n",
            "Epoch 18/20\n",
            "500/500 [==============================] - 12s 24ms/step - loss: 0.2436 - val_loss: 0.4702\n",
            "Epoch 19/20\n",
            "500/500 [==============================] - 12s 23ms/step - loss: 0.2398 - val_loss: 0.4793\n",
            "Epoch 20/20\n",
            "500/500 [==============================] - 11s 23ms/step - loss: 0.2384 - val_loss: 0.4812\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qZLEHtsg-X30",
        "colab_type": "text"
      },
      "source": [
        "The validation MAE stays in the 0.40s: you can’t even beat the common-sense baseline using the small convnet\n",
        "\n",
        "1D Convnet is blazing Fast and can be an alternate of rnn but they don't perform well here.\n",
        "\n",
        "We can also combine 1d Conv with Rnn to process large sequence that was not possible with just using Rnn.\n",
        "The conv-RNN will turn the long input sequence into much shorter (downsampled) sequences of higher-level features. This sequence of \n",
        "extracted features then becomes the input to\n",
        "the RNN part of the network.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a_9pRGiRBgYo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#####################################"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xLwV6JvoBOnI",
        "colab_type": "text"
      },
      "source": [
        "For longer sequences, you can either look at data from longer ago (by increasing the lookback parameter of the data gen-\n",
        "erator) or look at high-resolution timeseries (by decreasing the step parameter of the\n",
        "generator).\n",
        "\n",
        "Here, somewhat arbitrarily, you’ll use a step that’s half as large, resulting\n",
        "in a timeseries twice as long, where the temperature data is sampled at a rate of\n",
        "1 point per 30 minutes. The example reuses the generator function defined earlier."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZA2dmUMj-DRE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# previously step = 6 (10 min bt ka data hai, 6 step matlab 60 min(1 hour) ke bt ka data lo)\n",
        "# Now step = 3 (3 step matlab 30 minke bt ka data lo))\n",
        "\n",
        "step = 3\n",
        "lookback = 720\n",
        "delay = 144\n",
        "batch_size = 128"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Feg1P1KfC4-8",
        "colab_type": "text"
      },
      "source": [
        "# Model combining a 1D convolutional base and a GRU layer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sS2Si54LCz7g",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 367
        },
        "outputId": "e172c0b7-702d-46e9-df01-7e49db821975"
      },
      "source": [
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.optimizers import RMSprop\n",
        "\n",
        "model = Sequential()\n",
        "model.add(layers.Conv1D(32, 5, activation='relu',input_shape=(None, float_data.shape[-1])))\n",
        "model.add(layers.MaxPooling1D(3))\n",
        "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
        "model.add(layers.GRU(32, dropout=0.1, recurrent_dropout=0.5))\n",
        "model.add(layers.Dense(1))\n",
        "model.summary()\n",
        "\n",
        "model.compile(optimizer=RMSprop(), loss='mae')"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layer gru will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv1d_6 (Conv1D)            (None, None, 32)          2272      \n",
            "_________________________________________________________________\n",
            "max_pooling1d_4 (MaxPooling1 (None, None, 32)          0         \n",
            "_________________________________________________________________\n",
            "conv1d_7 (Conv1D)            (None, None, 32)          5152      \n",
            "_________________________________________________________________\n",
            "gru (GRU)                    (None, 32)                6336      \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1)                 33        \n",
            "=================================================================\n",
            "Total params: 13,793\n",
            "Trainable params: 13,793\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PcIV_FNSDDuc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 712
        },
        "outputId": "dafbd811-ac1e-40ff-9d1d-57019ca5cea2"
      },
      "source": [
        "history = model.fit_generator(train_gen,\n",
        "                              steps_per_epoch=500,\n",
        "                              epochs=20,\n",
        "                              validation_data=val_gen,\n",
        "                              validation_steps=val_steps)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "500/500 [==============================] - 140s 280ms/step - loss: 0.3432 - val_loss: 0.3214\n",
            "Epoch 2/20\n",
            "500/500 [==============================] - 139s 278ms/step - loss: 0.3064 - val_loss: 0.3010\n",
            "Epoch 3/20\n",
            "500/500 [==============================] - 139s 278ms/step - loss: 0.2880 - val_loss: 0.2969\n",
            "Epoch 4/20\n",
            "500/500 [==============================] - 137s 274ms/step - loss: 0.2792 - val_loss: 0.3005\n",
            "Epoch 5/20\n",
            "500/500 [==============================] - 135s 271ms/step - loss: 0.2695 - val_loss: 0.2903\n",
            "Epoch 6/20\n",
            "500/500 [==============================] - 136s 273ms/step - loss: 0.2626 - val_loss: 0.3006\n",
            "Epoch 7/20\n",
            "500/500 [==============================] - 135s 271ms/step - loss: 0.2554 - val_loss: 0.2963\n",
            "Epoch 8/20\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2484 - val_loss: 0.3016\n",
            "Epoch 9/20\n",
            "500/500 [==============================] - 134s 269ms/step - loss: 0.2418 - val_loss: 0.3136\n",
            "Epoch 10/20\n",
            "500/500 [==============================] - 133s 267ms/step - loss: 0.2360 - val_loss: 0.2999\n",
            "Epoch 11/20\n",
            "500/500 [==============================] - 134s 269ms/step - loss: 0.2309 - val_loss: 0.3032\n",
            "Epoch 12/20\n",
            "500/500 [==============================] - 135s 270ms/step - loss: 0.2266 - val_loss: 0.3217\n",
            "Epoch 13/20\n",
            "500/500 [==============================] - 138s 277ms/step - loss: 0.2224 - val_loss: 0.3145\n",
            "Epoch 14/20\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2192 - val_loss: 0.3157\n",
            "Epoch 15/20\n",
            "500/500 [==============================] - 136s 272ms/step - loss: 0.2153 - val_loss: 0.3125\n",
            "Epoch 16/20\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2113 - val_loss: 0.3223\n",
            "Epoch 17/20\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2086 - val_loss: 0.3193\n",
            "Epoch 18/20\n",
            "500/500 [==============================] - 138s 276ms/step - loss: 0.2075 - val_loss: 0.3163\n",
            "Epoch 19/20\n",
            "500/500 [==============================] - 135s 271ms/step - loss: 0.2041 - val_loss: 0.3246\n",
            "Epoch 20/20\n",
            "500/500 [==============================] - 139s 278ms/step - loss: 0.2011 - val_loss: 0.3238\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ekhRRAAwD0Nk",
        "colab_type": "text"
      },
      "source": [
        "Judging from the validation loss, this setup isn’t as good as the regularized GRU alone,\n",
        "but it’s significantly faster. It looks at twice as much data, which in this case doesn’t\n",
        "appear to be hugely helpful but may be important for other datasets."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0LdvVlwHEGIc",
        "colab_type": "text"
      },
      "source": [
        "#Wrapping up\n",
        "Here’s what you should take away from this section:\n",
        "\n",
        " In the same way that 2D convnets perform well for processing visual patterns in\n",
        "2D space, 1D convnets perform well for processing temporal patterns. They\n",
        "offer a faster alternative to RNN s on some problems, in particular natural-\n",
        "language processing tasks.\n",
        "\n",
        " Typically, 1D convnets are structured much like their 2D equivalents from the\n",
        "world of computer vision: they consist of stacks of Conv1D layers and Max-\n",
        "Pooling1D layers, ending in a global pooling operation or flattening operation.\n",
        "\n",
        " Because RNN s are extremely expensive for processing very long sequences, but\n",
        "1D convnets are cheap, it can be a good idea to use a 1D convnet as a prepro-\n",
        "cessing step before an RNN , shortening the sequence and extracting useful rep-\n",
        "resentations for the RNN to process"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vwJzEcxcEM5Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}